Fix MIL syntax + M1/M2 support - #6
Open
imperatormk wants to merge 4 commits into
Open
Conversation
The MIL scalar types used shorthand syntax (string("x"), int32(1)) that
only works on M4. Changed to the canonical verbose format that CoreML's
own compiler emits (tensor<string, []>("x"), tensor<int32, []>(1)).
Also targets program(1.0) with <ios16> instead of program(1.3)/<ios18>,
and simplifies buildInfo to just coremlc-version.
For conv-based kernels, adds runtime fp16 I/O fallback — M1/M2 ANE
doesn't support the cast op (fp32<->fp16), so on first compile failure
it retries with native fp16 inputs/outputs and does the conversion on
the CPU side. The fallback is persisted across exec() restarts.
Note: matmul and scaled_dot_product_attention ops still fail on M1/M2 —
these are M4+ ANE ops. The attention tests (test_ane_causal_attn,
test_ane_sdpa5, test_full_fused attention part) require M4 hardware.
Conv-based kernels (training, QKV projections, FFN) work on all generations.
Tested on M1 Pro, macOS 26.3 (Tahoe).
|
you need to fix the other files too not just the training dir files to make it work on m1/m2. |
Author
|
From what I see should be complete now although I'd like to fix attn too, would be keen to train a nice UNet like this |
codegen-sh Bot
referenced
this pull request
in dermitchell1993/ANE
Mar 2, 2026
Port upstream PR #6 (imperatormk) - fixes MIL scalar type syntax from M4-only shorthand to canonical verbose format that compiles on all Apple Silicon (M1/M2/M3/M4). Changes: - program(1.3) to program(1.0), ios18 to ios16 target - Scalar type shorthand to canonical verbose format - Simplified buildInfo dict (no M4-specific version strings) - fp16 I/O fallback: g_fp16_io flag with auto-retry on compile failure for M1/M2 where cast op is unsupported - Dynamic IOSurface byte calculation (bpe: 2 for fp16, 4 for fp32) Tested on M1 Pro, macOS 26.3 (per upstream PR author).
dev-erik
added a commit
to dev-erik/ANE
that referenced
this pull request
Mar 3, 2026
…program(1.0), ios16 target, tensor types across 18 files
ebowwa
pushed a commit
to ebowwa/ANE
that referenced
this pull request
Aug 4, 2026
… hardening #1 /models bypassed safety: now uses services.safe_compile (lint → validate → passes → compile) instead of calling bridge.compile directly. The shared compile service enforces all /raw/kernels safety rules (MIL lint, positive tensor-size validation, tensor-count limits, compiler passes, weight-name validation, bridge-error mapping). #2 Model leases leaked: register_model now does atomic replace (free old lease before installing new). unregister_model calls bridge.free(kernel_id) before removing the dictionary entry. maderix#3 Async eval bypassed admission: now calls admit_eval BEFORE dispatching the background task, and finish_eval in _bg's finally block. max_pending backpressure applies to async work. maderix#4 Cancellation resurrection: _bg checks sub.state == CANCELLED before transitioning to EXECUTING. A cancelled submission is not resurrected. maderix#5 Submission memory leak: SubmissionRegistry now has max_retained=100 + ttl_s=300. _prune() evicts terminal submissions by TTL and count on every create(). maderix#6 Batch-eval bypassed admission: each item now goes through admit_eval/finish_eval. Endpoint honestly named "serial batch" (NOT parallel without kernel replication). 138 tests. Co-Authored-By: Claude <noreply@anthropic.com>
ebowwa
pushed a commit
to ebowwa/ANE
that referenced
this pull request
Aug 4, 2026
…models admission #2 W&B: _process now handles event/run_finish (previously discarded). Fallback delegates correctly when wandb absent (metrics + traces + events forwarded). finish_run finalizes in queue order (no race). flush() drains with 5s timeout. maderix#3 LocalJSON: writer calls task_done() (flush no longer deadlocks). maderix#4 Backend registry: constructed in lifespan (ANEBackend + MLXBackend), stored on app.state.backends. maderix#6 /models: run_model now goes through admit_eval/finish_eval (previously bypassed scheduler). register_model uses safe int parsing (try/except → 400, not 500). maderix#8 /v1/resources: uses app.state.settings (was always false), ANE_PUBLIC_ENDPOINT env (was guessing hostname), correct tailscale_serve detection. Lifespan sets app.state.settings. Lifespan flushes all observability backends on shutdown. mlx has platform marker in requirements.txt (Darwin arm64 only). 149 tests. Co-Authored-By: Claude <noreply@anthropic.com>
ebowwa
pushed a commit
to ebowwa/ANE
that referenced
this pull request
Aug 4, 2026
…, W&B fallback, deadline parsing #1 All integration routes (/mlx/compare, /mlx/diff, /pipeline) now go through admit_eval / finish_eval — no route bypasses the scheduler. Pipeline ANE steps also admit. The /mlx/compare demo uses try/finally to guarantee kernel lease release. #2 Batch-eval validates inputs_per_item >= 1 and rejects input counts not exactly divisible (no silent tensor discard). Both deadline_ms parses (eval + batch) now use try/except → 400 for non-numeric X-Deadline-Ms (was 500). maderix#3 W&B fallback: when wandb is not installed, log_metrics / log_event / log_trace delegate directly to the fallback instead of queueing into an unconsumed queue (previous code started no uploader thread without wandb → events stuck forever). maderix#4 SubmissionRegistry.mark_terminal() calls _prune() immediately when a submission completes (no waiting for the next create()). maderix#6 test_models_register_rejected_by_lint: skips when ANE bridge unavailable (was asserting 422 on non-ANE hosts where /models returns 503 before reaching lint). Pipeline spec JSON errors now return 400 (was 500). 149 tests. Co-Authored-By: Claude <noreply@anthropic.com>
ebowwa
pushed a commit
to ebowwa/ANE
that referenced
this pull request
Aug 4, 2026
… mx.distributed, full output descriptors maderix#6 Remote store qualified references: put/get/list/delete now use tier-prefixed IDs (remote:<host>:<tid> or local:<tid>). A failed remote put that falls back to local returns a local: ID; get/list/delete route to the correct tier based on the prefix. list() merges both tiers. The namespace is coherent — no silent namespace collision between machines. maderix#7 Deleted stale mx.distributed path: removed ane_distributed.py, mlx_worker.py, and test_distributed.py. mx.distributed only supports single-machine multi-core; the TCP worker (mlx_tcp_worker.py + ane_tcp_transport.py) is the real cross-machine path. The distributed endpoints were already removed in the prior commit; now the dead code is gone. maderix#8 ANE backend output descriptors: load() now stores full output_descriptors from the ArtifactDescriptor spec (dtype + shape + byte_length) per executable_id. execute() constructs output TensorDescriptors from the stored specs, not invented [1,N,1,1] shapes. unload() clears the specs. Callers who provide output_descriptors in the artifact spec get exact output metadata; callers who don't get byte_length-correct descriptors with empty shape. 7 tests (updated for qualified refs + merged list + fallback). 156 total. Co-Authored-By: Claude <noreply@anthropic.com>" git push -q origin ane-compute-api && echo "pushed"; git rev-list --count origin/main..HEAD
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MIL scalar types used shorthand syntax that only compiles on M4. Changed to the canonical verbose format that CoreML's compiler emits (
tensor<string, []>("x")instead ofstring("x")), targetsprogram(1.0)/<ios16>, and simplifies buildInfo.For conv kernels, adds runtime fp16 I/O fallback — M1/M2 ANE can't do the
castop, so on first compile failure it retries with fp16 inputs/outputs and converts on the CPU side.Tested on M1 Pro, macOS 26.3.